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DETAILED ACTION 
Claim Rejections - 35 USC§102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under (his section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in ( I ) an application for patent, published under section 1 22(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in die United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

2. Claims 1-3, 6, 9, 14-16, 19, 22, 27 and 30 are rejected under 35 U.S.C. 102(e) as being 
Anticipated by Gurevicta (US. 6,499,036). 

3. Regarding claims 1-2, 6, 9, 14-16, 19, 22, 27 and 30, Gurevich discloses a computer 
system including software and, implementing method, in which capable of providing distributed 
functionality to a plurality of clients comprising: a first provider server having a function 
provider module therein (Fig. 2, 210); a data store connected to the function provider module and 
containing function information defining at least one function object, each function object 
associated with a function and comprising a function name element and specifying at least one 
parameter (DOM 115); the function provider module being configured to: in response to receipt 
of a first type request from a requester, return a set of function objects to the requester (Fig. 9, 
662, illustrated list of functional objects in response to first type request); in response to receipt 
of a second type request from the requester containing a function object having defined 
parameter values stored there, evaluate the function associated with the function object (Fig. 9, 
910), modify the received function object to include results of the function evaluation, and return 
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the modified functi n object to the requestor (return result 900). Regarding claims 2, the system 
of claim 1, wherein each function object further comprises a function ID and an identity of a 
respective provider server (see example Fig. 10-1020; function GetBankCaid comprises script 
which include ID and identity, "BIN", SID). 

4. Regarding claims 3, Implicitly, Gurevich discloses the data can be returned from any 
appropriated data source (130). Although Gurevich does not explicitly state a third server or so 
forth, but it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made that Gurevich inventive concept can be replicated to any number servers or 
service providers as desirable. 

5. Claims 4-5, 7-8, 10-13, 17-18, 20-21, 23-26, 28-29 and 31-32 are objected to as being 
dependent upon a rejected base claim, but would be allowable if rewritten in independent form 
including all of the limitations of the base claim and any intervening claims. 

6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Bunjob Jaroenchonwanit whose telephone number is (571) 272- 
3913. The examiner can normally be reached on 8:00-17:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David Wiley can be reached on (571) 272-3923. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status informati n for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-directuspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBQ at 866-217-9197 (toll-free). 
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Compiling Distributed C++ 



Harold Carr t Robert It Kessler, Mark Swanson 
Department of Computer Science 
University of Utah 
Salt Lake City, Utah, 84112 



Abstract 



2 Domains — Abstract Processors 



Distributed C++ (OC++J is a language for writ- 
ing parallel applications on loosdy coupled distributed 
systems in C++. Its key idea is to extend the C++ 
class into S categories: gate w a y dosses which act as 
coram ttnication and synchronization entry points be* 
twee n abstract pr ocessor s, datsts whose instances may 
be passed by value between abstract processors via gate- 
ways, and vanilla C++ classes: DC++ code is com* 
piled to C++ code with calls to the DC++ runtime 
system- The DC++ compiler wraps gateway classes 
with handle classes so that remote procedure calls ore 
transparent* It odds static variables to value rlatt tt 
. and produces code which is used to marshal and cto» 
marshal arguments when these value dosses ore used 
tn remote procedure calls. Value dosses are deep 
copied and preserve structure shoring. This 
shows DC++ compilation and performance* 



1 Introduction 



DC++ b designed to exploit loosely coupled dis- 
tributed systems built fay intercoeinecting multiple 
workstations through a local area network. DC++ is 
a distributed version of C++ (*] (this paper awrames 
knowledge of C++). DC++ provides a small num- 
ber of simple wrtmsions to C++: 2 built-in Hsssrr 
DeDoaaia and DcThraed; and 2 new categories of 
class i rat mice usage: gateways bet ween domains, and 
"value" instances which may be passed between do- 
mains through gateway member function invocation 
sod return. The DC++ language is discussed in [S, 6]. 
This peper abowa how DC++ b compiled. We win use 
the bou nd e d buffer problem [1] as a running example 
throughout this paper. We include performance mea- 
surement! for tab example. 



DC++ supports parallelism by providing 2 types: 
domains and threads f (13, 17]). A domain is a logi- 
cally encapsulated address and control space, an ob* 
street processor. A domain is similar to a monitor [11}: 
they ensure mvtvd exclusion synchronisation by en- 
forcing the rule that only one thread of control may 
be active in a domain at a time. If another thread at- 
tempt* entry to an occupied domain, that thread will 
be queued on a FIFO queue for later entry when the 
domain becomes vacant. It differs from a monitor in 
that domains may be dynamically created and deleted. 
Further, a domain by itself does not have any entry 
points* Entry points may be dynamically created and 
dyjftfd by creating gatewa y class instances into do- 
mains. A domain b created by specifying a physical 
processor number (O-based) on which to allocate the 
domain. For the bounded buffer example we will cre- 
ate 6 domains* one each for the 3 producers, one for 
the buffer, and one each for 2 co nsumer s: 

const SBB_prodac«r« • 2| 

coast ihiQ unaueais • 2; 

DcDeaaln* pdCnaa_prodac*ra] t 

DeDoaaia* odCgga^consuwa] ; 

DcDeaaln* bd ■ mi DcDoaaiataavpcooucore ♦ 

for (ins i • 0; i < ruBa_proouc«ra; 

pdCi] - nmu DcDt»in(i); 
for (i - 0; 1 < mB.cou»Ti; *♦!) 

cdtO ■ tm DcDoaaiad ♦ an^raeacara) ; 

Since there may be more domains than actual phys- 
ical p r ocess ors , the domain is allocated oo i X 
OcHodoCountO, where DcJIodeCouatO returns the 1- 
bssed «*«Tnlw of physical prorraanra available to the 
specific execution. Tarn means that more than one do- 
main may be explicitly (by the prc^rainmer)ceuimlic- 
" lUy (by the domain allocator) created on a processor. 
The DC++ runtime system supports multitasking so 
the programmer need not be concerned with these de- 
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3 Gateways — Domain Entry P into 

A gateway b a system-wide unique "pointer" to an 
object created in a specific domain. It b treated as 
an ordinary C++ object: member function invoca- 
tions on gateway objects result in remote procedure 
(RPOs) if that gateway resides in a different do- 
main than the domain from which it b invoiced. Oth- 
erwise a vanilla C++ member function invocation re- 
sutts. Since RPCm look identical to vaniUa C++ num- 
ber functto invocation 

possible without program modification (modulo syn- 
chronisation characteristics of the algorithm). 

A gateway class b declared like a vanilla C++ dan, 
except it inherits from DcCatevay: 

class Producer t public Dcustevmy < 

int naeuitems; // tuber items to produce. 

Batter* buffer; // Buffer to put the* in. 
publics 

Producer (int i. Buffer* b) { 
mxm_it**s - 1* buffer ■ b; 

> 

void Produce 0 < 
shile (nu».lt«»*--) { 
b«ffer->Oeposit(nes Derived (num.it ems))s 

> 

> 

>; 

class Buffer t public DcOstevay { 
unsigned aw*.ltem*t // Capacity of buffer. 
Derived** slots j // Array to contain items. 
-"- < f~ t bead; // wmere producor puts items, 
unsigned tail; // snare consumer gets Items, 
t-^ I^H sIm; // foaber of items in buffer* 

publics 
Buff«r(iiit U< 
mead • tail • slxe ■ 0; 
max_items - ii 

slots • new Derived*0sax.itee*Ot 
makeDeUTQueue (Buffer: sOeposit) ; 
DQOpen (Buffer: {Deposit) ; 
mtte&elajQuem (Buffer: sFetdO ; 

> 

void Deposit (Derived* i)( 
slots [tail] -.i; 
sixers 

tsU • (tall ♦ 1) X ■u.itm; 
if (else > 0) 
DDSpen (Buffers i Fetes) i 

> 

Derived* Petch(){ 
Derived* retval - olots Ch o ed ]: 
a is*— j 

bead - (bead ♦ 1) X max.it*** { 
if (eiae < max_it«m*) 
DQDpemCBmfferi t Deposit) i 



return retval | 

> 

>; 

class Conen—x : public DcOatevsy i 
int mus.it«*s; // sumber items to constat*. 
Buffer* buffer; // Buffer to get tha tram. 

public: 

Consumer (int i. Buffer* b) < 
sam_it*ss • I, buffer • b; 

> 

void ConsumeO < 
vnile tanuitem*--) € 
Derived* i - buff er->Fetcn(); 
if (i->0stsO — 0) 

return; 
epimO; 

> 

> 

}; 

The butler has explicit delay queues a sso ciat ed with its 
Deposit and Fetch methods. Delay queues provide 
condition tynchromxation. If the delay queue is open, 
calls to the method proceed as normal. If the delay 
queue is dosed then the calls are queued on a FIFO 
queue for later entry to the method when the queue is 
opened. A call (thread) waiting on a method's delay 
queue is not considered to have entered the domain. 
They are used in this example to crsmre thai producers 
only deposit items when there is room in the butter, 
and that consumers only fetch items when there is one 
or more available. 

A gateway rnrtanre is created in a specific domain 
by providing an optional domain argument as the first 
argument to the gateway's constructor. If not pretest 
the current domain is used: 

Buffer* b - mee Buffer (bd. 8)t 
Producer* p tntuuproducera] ; 
Consuser* c Cwm.i r 1 1 1 1 ■ i» e rs ] ; 
for (l - 0; 1 < ima_prodacers; «*i) 

pCi] • mev r^oducerCpaCa. ». b)s 
for (i • 0i i < mum.com*«mersi ++i) 

c(U - nes Consumer (cdt 13 » 26, b); 

Note thai the tn* passed between the producer, 
buffer and coiisumer is Derived*. This type is a user 
defined "value" dees and will be cSscusted later. Abo 
note that the optional domain argument is not de- 
dared in the user's gateway dams definition. This b 
handled by the compiler. 



4 Threads 

Concurrency b achieved by creating multiple 
threads of control. In the bounded bufter example, 
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thread* are created for the producers tod consumer*, 
whereas the buffer b passively enclosed in a domain 
to ensure mutually exclusive Access to it: 

for (1 ■ 0; 1 < imsucmisnmors; 

me* DcTnread(cUj->C*a*»0)i 
for (1 ■ 0; I < sssuprotecara; 

now DcTbx«ad(p[U->rroduceO); 

Threadi may return values when they terminate. 
These values may be used by the thread which created 
the new thread, and/or the termination of the crested 
thread may be dVirct^d by the creating thread. This 
feature is not used in this example. 



5 Compiling Gateways 

The fundamental idea is to transform the program 
so that all references to user defined gateway classes 
are changed to references to compiler generated han- 
dle dosses* Each user gateway class has an sssoriitrd 
handle class which intercepts all method invocations. 
The handle class determines if the gateway method 
being invoked is for an instance in the same domain 
as the invoker. If so, it does a normal C++ method 
invocation. Otherwise, it marshals the arguments and 
does an RFC to the domain in which the gateway re- 
sides. Upon return It onmarshafs the return value. 
The compiler generated handle class frees the pro- 
grammer from these details and allows gateways to 
be redistributed without program modifies tion This 
section shows the details of the gateway compilation 
process* 

A unique tag is created for all gateway classes in a 
program 

•ana CU8S.TA68 < 

OonsitsMr.Taa. 

Baffor.TaO, 

rrodttCsr.TAQ, 

>; 

These tagi are used to indicate the type of object being 
made when that object is created remotely. 

Each user gateway class is wrapped by a handle 
das*: 

class Suffer.*' : public gateway < 
public: 
BnfferJKl&t i> 

i getewayCaev Buff or (1)1 O 
ftaffer.VCDcdQeain* 4. iat i) 
i gateway<d, 

RakeJIemoteOb ject (Buf f er.TeO , 
d. i)> O 



Derived* FetcfaO; 

void Deposit (Derived* >t 

>; 

For each constructor in the user class, 2 constructors 
are created in the handle class: one with a type signer 
ture identical to the user's definition, and one which 
adds a domain argument as the first parameter. In this 
way gateways may be created in the same domain or 
between domains. 

The handle class inherits from gateway, whose def- 
inition is: 

class gateway { 
private: 

DcDemain* domain: // domain where object lives 
DcQojsct* remote; // remote object 
void* local; // local object 
protected: 

DcDemaiao domaiaOidO < return domain; > 
DcGbject* remotcGbjO < return remote] > 
void* localObjO < return local; > 
void* localPO { return local; > 
gateway(void* v) : 

doaain(IUU) v remote CWU), local (v) {> 
tatewayOcDomaifi* did* DcObject* eld) : 

domain(did), remote(oid), local tiULL) .O 

>; 

When creating objects derived from gateway, a gate- 
way is returned which either points to an instancy in 
the domain or to an instance in a remote domain. 

When the handle darns constructor is given an 
optional domain it constructs the actual object 
on a remote node via the compiler generated 
K ako ft enotoObjoct routine: 

DcObjoct* MakoKomotoObjoct (unsigned typeX 
void* v * IUIXs 
switch (typo) i 
case C oo mom or ,Tag : 

v • new Consumer; break; 
case Buf f or.TAG s 

v • mew Buff or j break; 
case rrodttcor.TAQ : 

v • mew Producer; break; 
dofaslt: 

Dc£rror( i n2Bkaevii object typo**); 
break; 
> 

return Iegiater8bject(v t DcTftUDomainQ); 

> 

Regis terObject is a DC++ runtime system routine 
which places the actual pointer to the newly created 
object into an OutTable table, a table of entities which 
may be used remotely. It returns a DcObjecte which 
is a special pointer type which is unique and valid 



between domains. Hie bit* of tbii pointer indicate 
the node oo which the object resides and the index 
of the actual object pointer in that node's Dutiable 
table. 

This example is incomplete In that it doesn't show 
how arguments to constructors are handled remotely, 
and it only shows one constructor per class. If there 
is more than one constructor they axe distinguished 
by their type signature. Constructor arguments are 
handlfd in a n , » n "» r similar to arguments to m ri h ods 
(shown below). 

A tag it created for each public method in the han- 
dle class (constructor tags not shown in example): 

*am BttffermjETHDOJTACS < 

D*xiv«4ptrB»ff*xr«tch.TaQ, . 
ToldBuff«xf>«po«itD*rtv*dptr_Ttf, 

>; 

For each method in the user's original gateway def- 
inition an aiwofiatfd mrthfy* is defined in the handle 
dees: 

void ftttferjh :DcpositCDtrived* a3){ 
if (localPO > 

return ( (Bur ier _V«) localOb j 0 >->Ocpo«lt (»3) ; 
JUfftutt* f - llalteKsgBuf (100) ; 
S*tHagBuf(f, O. r«aot«CbjO)l 
SetftagBuftf, 1, voldMf *r&*poaltDtriv«dptr_T4G) : 
PlCX_voijaMait*xf>*pa*i&*rU*a^JUa& (f , e*3); 
il^T«lrhlTrf>n«iln(aaCTEJttff<rJW*nHfa t 
x, denainaidO); 

l>eleteltagBaf(f)i 

> 

Derived* Buffer.*: :Fetch(H 
if (localPO) 

return ((BufferJN>localCbJ(»->FetchO; 
HagBofld f - tek*Jlsgfeif(2>| 
SetRegBufCf, 0 # reaoteObjOli 
fletHsgVufCf, 1, Oeriv«4ptrBiaff«rFetch.Ti0); 
Ksjpuf Id fr * (HsgBufld) 

t. dcaaiaflidOh 

DaleteHsgfiufCr): 
Derived* result; 

IVaOUtariv*n>trBaf fexFetchJuTVlUfr,, ereeult) ; 

DeleteSsgPofCfr): 

return result t 

> 

These handle methods are how transparent RPCs are 
achieved. Ef a Buffer handle instance is created in the 
same domain as the inroker of its constru ct o r then a 
pointer to that local object is installed in the gate- 
way's local method variable. When a handle method 
■ invoked U n^ checks to see if the object b lc<al. If 
so it avoids the overhead of RFC by invoking the local 



method. Otherwise it marshab the remote object ref- 
erence, the method tag, any arguments, and then calls 
the RPC handler in the domain for that handle dan 
via ApplyVithinDoaain. ApplyVitfcinDoaain is the 
blocking remote procedure call routine in the DOM- 
runtime system. When the RPC returns, it unmar- 
shals the return value into a message buffer. The op- 
eration of compiler generated PACK and UBPaOC mar- 
shaling routines is discussed later. They use runtime 
message buffers (KsgBaf Id) to contain object refer- 
ences, method tags, argument values, and return val- 
ues. 

A remote method handler is created for each handle 
class. The method tags are used to dispatch to the 
appropriate method when handling RPCs: 

void* KWTiLeuff«rjunw(llsgBufId r)< 
Buffer* r - (Buffer*) OatTableCRsgPuf (f , 0))| 
BofferPTOJuTTHDD.TaCS a - HsgBuf tt . 1): 
switch (a) { 

case Darivedptrluff crP«tc&.TAfl i < 
DeleteftsgPufCf); 
Derived* s4 - r->F«tch() i 
fUetof Id f r - 

PaCXJWrivedptrBnf f erF*tchJtETVAL(aa4) ; 
return (void*) fr; 

> 

case voidSufferDepoeitDerlvedptr.TAQ t < 
DeieteXsfBufCfi: 
Derived* aSs 

QIPiCK.v«iitSuf f erD*pealtDerivedptrJ>ABMSa , 

taS); 

r->Deposit(a3)s 
return sHU.; 

} 

default: 
Dc£fTor(*U*noen a«thod»>: 

> 

) 

This routine dispatches to the appropriate method 
call, marshals the arguments from the m es sag e buffer 
and calls the neeociatari user operation. Return values 
are placed in a message buffer to be used on the other 
end of the RPC. 



6 Value Objects 

Systems such as [21 provide primitives for send- 
ing and receiving data between processes, but require 
the programmer to pack and unpack aggregate data. 
Besides the gateway daaaes, which marshal and nn- 
matahal argument* and return value*, DC++ pro- 
vides •value" classes so that programs may pass deep, 
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structure-preserving copies of value class instances be- 
tween domain*. A value object u a d&M which inherits 
from the built-in DC++ class DcValua: 

class B«m : public Detain* < 
lat data} 

public t 

BaaaCint 1 - 0) : data(i) { > 
lat DataO I return data; > 

>; 

class Contained i public DcValua < 
char c; 

public x 

CoatalnedCchar .c • 't') : c(.c) { > 

U 

class Derived x public Baaa < 

Contained ai, *apl t *ap2; 
double d; 

public: 

DerivedCiat .1 • 0. 

double _d ■ 0.0, 
char cl • V. 
char c2 - •*») 
x Ba»«(.i). 

mi(cl). 

dCd)» 

■pi Cn«« Containad(c2) ) , 
ajtfCspl) C > 
"DerivedO I dalata apt; > 

>; 

(Note thai member data apt and up2 point to the 
same instance.) 

Inheriting from DcValua indicates thai when one 
of these object* ta passed as an argument to, or re- 
turn value from a gateway member function, it t* to 
be totally copied. This mean* thai any objects, point- 
er* to objecta, or built-in data types contained within 
the passed object, must be recursively copied and any 
structure sharing present via pointers must be pre- 
served. Any member data object* or member data 
poi n t er s to objects, must be objects which also derive 
from DcYalue so the compiler can add the necessary 
support for the complete copy operation. The com- 
piler handles OH- scalar types automatically. 

Value objects may be psssed-by-value freely be- 
tween domains via gateway member function argu- 
ment* and return values. This is seen in the exam- 
ple by the calls to the Buffer methods Deposit and 
Fetch which accept and return instances of the user 
defined Derived class. These routines actually pass 
end return Derived*. In this case, the object i* still 
copied and the pointer to the newly created object 
on the receiving end is used rather than the original 
pointer. 



7 Compiling Value Objects 

7.1 Runtime Type Information 

The compiler adds static variables and virtual func- 
tions to each user class which inherit* from OcValue: 

daaa Base x public DcValua < 
DM JWLTTPBCBaaa) ; 
Int data; 

public: 

Baaatint i - 0) ; data(i) < } 
int DataO < return data; > 

I; 

The D8CL4BJLXVPS macro: 

Sdef in* OeCUu^TTreCaw) \ 

public: V 

virtual conat Typalafo* TypeO coast \ 

< ratura Blafe.objs > \ 
static const Typalafo* InfoO \ 

< ratura tiafo.obj; > \ 

virtual void tfrit«r<oatraamt) canst ; \ 
static DcValua* Baadar(i*traaaa) ; \ 
statu name* B*ad(lstreent a> \ 

. < return (n**e»)DcYalua: :SU*d(«) ; > \ 
aaaeCiatresat); \ 
private: \ 
static coast Typalnfo lafojobj \ 

defines Type and Info mrlrtnds u*ed to obtain type in* 
formation at runtime. This information is contained 
in the private static member data info job j. Type is 
used to get this information given any imtsnce which 
derives from Devalue (eg., soa*_in* t*nc*->Typ* ( ) ) . 
Info b used to get this information if the type is 
known before hand (e.g., Base:: InfoO). These are 
used to obtain type-specifk reader functions used 
when sending objects between domains. 

7*2 Sending Objects Over Streams 

The Writer, Reader, Mad methods and the 
naaaCiatraaaa) constructor are used to convert ob- 
jects to/from linear byte streams for transmission and 
rec ep ti on between d oma ins, The compiler generates 
these for each type that directly or mdirectly inherits 
from DcValuo: 

DEnULTTTBCBa**) ; 
lUVlllLTYfC (Gontainad) ; 
DEPIlE.TTTE(Darivad) ; 

The D2FI5XJTF8 macro: 

tdaf ina DEPIBB.TTPB(aa>a) \ 
Devalue* saaai sBaadarClatraaaS a) \ 
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< return bn um(i) ; > \ 
const Typalnfo iiaa4::infQ.obj(STftII0IPY(naa«) , \ 

tmii xBaadar) \ 

defines the Rudtr method which uses the constructor 
(torn istrean to create «n instance of the class given 
a linear eiream of bytes. It also initializes the static 
lafo^bj method data to contain the name of the class 
and a pointer to the dew reader. 

The compiler generates a Writer method and an 
ietreaa constructor for each value class: 

Bases sB*aa(i»tc«a»t •) : DcYalo* (•) < 
a » data; 

) 

void Baa*: : Writer (estreaat a) coast < 
a « data « endl; 

> 

Coataised::Oontained(istreaat a) : Devalue (a) < 
a » c; 

> 

void Contained:: Writer <ee*reaaft a) coaat ( 
a « c « «ndl; 

> 

Derived: : Derived (istreamt a) 
: BeeeCe), 
ai(e>. 

apt (Contained: : Bead (a) ) , 
ap3 (Contained: :Bead(a) ) 

< 

a » dj 

> 

void Derived: : Writer (eatreaas; a) const <' 
Baae::ttriter(a>; 
■i.WriterU)i 
apt->Write(e); 
ss>2->Vrite(a); 
• « d « audi; • 

> 

Derived types first call the static Writer for their 
base classes (only one in this example)* Then they 
write oat instances using the Writer method of the 
fnitnmn Pointers to instances are written using the 
Write method. Write keepa a table of pointers to pre- 
serve structure sharing. Built-in data types are writ- 
ten and read to and from a stream with the normal 
C++ loatreaa operators. Data b written and read 
in mVntical order* 

7«£ Argument Marshaling 

The compiler generated Buf far Jf: : Deposit 
method receives a Derived* as an argument. It con- 
verts this to a hnear byte stream via the generated 
PACa^jroidnu»erDepoeitDerivedptrJ > AHXS routine. 
This routine uses the write operations to write into an 
output string stream: 



void PaCX.void9ufferDepoeitl)erived>trJ k Aail3( 

KigBufld t. Derived** a3>{ 
oatratreaa at rout; 
(«a3) •>Vrit« (atrout) i 
FackltagBuf (streat .etr 0 ) ; 

> 



Write puts a linear representation of the contents of 
the instance of Derived into the string output stream 
atrout. The byte stream component of the stream b 
obtained with the C++ standard atrstreaa.h rou- 
tine s trout. str(). This b given to the message 
buffer for transmission to another domain (potentially 
over a LAN). (It b the message buffer's job to delete 
the byte stream when it is no longer needed.) 

On the receiving end, a£X0TILButter.HAXDU9l uses 
UNPACKjroid^iilterDapoaitDerivedptrJ'AfillS 

to convert this byte stream representation back into 
an instance in memory: 



void Ul7ACK.voi<iBarXerO«poaitD«rivo4ptr J'itKS ( 
KsgBufld x. Derived** U>< 
iat* fb • RagMBaaett)] 
char* fbp - (char*) (fb ♦ 3); 
iatratreaa inatr (fbp. atrlen(fbp) ♦ 1): 
•aS ■ Derived: tEead(inatr) t 

> 



The message buffer'a byte atream b obtained via 
EagBuf Bane. It skips past the first two words (which 
contain the object refe r ence and method tag) and uses 
the resulting stream and its length to create a stan- 
dard a tratreaa.h iatratreaa (input atream airing). 
Thb stream b given to the static Derived class stream 
reader. 

Derived : : Bead gives 
the stream to the input stream constructor for that 
type Derived : : Derived (ietreaa*) (defined above 
by DSFXHRJTTFB). Thb in turn passes the stream first 
to the Base constructor and then the Contained con- 
structor via Base(s) and sd(a) respectively. Since 
the next two fields, apt and ap3 are pointers, these 
are initialised from the stream vUexplfchcalb to their 
static reader Djetbods: Contained: :Bea4. These ex- 
plicit calls are necessary to detect structure sharing, 
whereat constructors are called when the method data 
contains an instance rather than a pointer. In thb ex- 
ample, apt and ap3 do point to the same structure 
on the sending end, so thb will be prtau v td on the 
lecesving end. The leader routines keep a table of 
pointers for thb purpose. 
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8 Performance Measurements 

These measurement* were Uken oo networked HP 
Series 9000, Model 370 Workstations. Commuiucation 
between pioceosotB is via BSD sodceta. For the mea- 
surements we created three producer thread*, a sin- 
gle buffer, and two consumer threads. The producers 
produce 25 items each- The buffer capacity is eight. 
The consumers con Bnmc until they have consumed 25 
items, or the Data item is 0. The consumers spin (rep- 
resenting some work) for a period before attempting 
another fetch, like allocation of domains/gateways 
to proces sor s, sod the mte r proce ss or communication 
pattern is: 




pN, b and cN represent the producer, buffer, and 
consumer domains and their associated gateways. The 
circles r epresen t real processors. The arrows represent 
caDer/callee patterns. 










































































































































































































































































































The base case for both- is lor the producers, the 
bufler and the consumers to til multitask an s> sin- 
gle processor . Speedup is achieved at three processors 
once the bufler » allocated to its own processor. At 
that point the consumers do not have to wait for a pro- 
ducer multitasking with the buffer to obtain access to 
the buffer. At four a slight speedup is achieved, but 
the producers must wait for the consumers to open 
space in the buffer. The best speed is obtained at five 
procejsocs when the consumers each have their own 
pi o c e sso i. Adding another processor slightly increases 
the time since the producers produce faster than con* 
sumers can consume. The increased time b due to 
in* 



Other alloca t ion patterns are possible, but are not 
the subject of this paper. Automated resource allo- 
cation is bandied by [9]. Other problems such as the 
dining philosophers problem programmed in DC++ 
have shown realistic speedup [5]. 



9 Other Parallel C++ Languages 

Another way to provide concurrency is to define 
an abstract class, Task, that implement* the task ab- 
straction. The constructor for Tank creates a thread 
to' animate the task. User-defined task classes may 
inherit from Task. This approach has been used to 
provide coroutine facilities [16, 14] and simple parallel 
facilities (7, 3, 4). 

This "active object" approach has problems. (1) 
Code to start the task body appears in Task's con- 
structor. In C++, that code i* executed first, before 
the constructors of any derived classes. Therefore, the 
new thread must be created in a blocked state, and 
must be unblocked after derived constructors finish. 
(2) Placing the task's body m its constructor is ruled 
out by inheritance, since a derived class must still ex- 
ecute the private data initiafisation of its base class 
but override the base class task body. A solution is' 
to make the body a virtual naln member function so 
derived tasks can replace it. (3) During initisfintkra, 
while Tank's constructor is executing, the new task Is 
considered to be an instance of dam Tank, not the ac- 
tual derived task class being instantiated. Therefore, 
calling main in Tank's constructor will not execute the 
correct task body. DC++ avoids these problems by 
using "passive* objects. In met, DC++ unbundles 
everything: threads are distinct from domains, which 
are themselves distinct from objects and gateways. 

DC++ gateways (which provide RPG) are similar 
to "handles* in ESP [15]. However, use of ESP handles 
may requite casting and may be visible to the pro- 
grammer, whereas type-safety and the tnrchairins of 
gateways are bandied by the DC++ cwnpfler . CC++ 
[13] uses a co mbin ati on of ^global pointers" and Tog- 
ical processor object** to provide RPC. CC++ also 
employs single-assignment "sync* variables and the 
notion of "atomic* functions, whereas DC++ uses do- 
mains for synchronisation and atomicity. 

Systems such as (2], require the programmer to 
marshal values passed to remote procedures by hand. 
DC++ extends C++ such that values passed between 
domains are automatically (un)marsh ailed. DC++ 
extends this capability to user defined types through 
the concept of "value* classes, which may use all tn- 
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heritance mecfaaoimu. DC++ value types an similar 

to ObjecUO in the NIH CUss Library [10]. 



10 Conclusions and Future Work 

We have designed a distributed version of C++ 
based on concurrency mechanisms developed in our 
previous work in Concurrent Scheme [18]. We are con- 
tinuing development of the DC++ compilef so it will 
folly automate the compilation process and detect il- 
legal DC++ usages. TheDC++ runtime system runs 
on homogeneous workstation*. We plan to extend it 
to handle heterogeneous systems. The current perfor- 
mance measurements ate very encouraging. 
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